Effectiveness of a Novel Tablet Application in Reducing Guideline Deviations During Pediatric Cardiac Arrest

Key Points Question Does the use of a tablet application cognitive aid reduce deviations from American Heart Association (AHA) resuscitation guidelines and improve the management of pediatric cardiac arrest? Findings In this randomized clinical trial including 300 participants in 100 teams, the use of an interactive tablet application led to fewer deviations from AHA guidelines and better team performance compared with use of the AHA pocket reference card or no cognitive aid. Meaning This randomized clinical trial found that the use of the cognitive aid tablet application improved adherence to resuscitation guidelines, thus demonstrating promise for improving patient outcomes, although further studies are necessary to confirm these findings and demonstrate its impact in clinical practice.


Introduction
Pediatric cardiac arrest is a rare emergency associated with high mortality and important clinical sequelae. 1,2 Deviations from international resuscitation guidelines often occur and can negatively affect patient outcomes. [3][4][5] Electronic and paper-based tools have been developed to support resuscitation teams in optimizing the management of cardiac arrest, but most of them are focused on adult cardiopulmonary resuscitation (CPR) in an out-of-hospital setting 6 or on the quality of chest compressions through audio and/or visual feedback. 7 Several electronic cognitive aids have been developed to improve adherence to resuscitation guidelines; however, most of these tools are not tested in terms of content, usability, and users' perceived workload. 8 To our knowledge, no realevent study 9 and a limited number of simulation-based studies have tested the impact of cognitive aids on the management of cardiac arrest. Most of these studies focused on adult case scenarios, included a limited sample size, and presented important methodological limitations. 10 In 2019, we developed an interactive, multimodal, electronic cognitive aid in the form of a tablet application (app), named PediAppRREST, to provide decision support to the team leader through a clickable list of actions to be performed in a stepwise manner, closely following the American Heart Association (AHA) resuscitation guidelines. The app was developed and refined through an iterative prototyping approach on the basis of users' needs and feedback 11 to overcome the most frequent deviations detected in simulation-based observational studies. [12][13][14] The objective of this study was to test the effectiveness of the PediAppRREST app in reducing deviations from AHA resuscitation guidelines during the management of simulated pediatric cardiac arrest.

Methods
This randomized clinical trial was deemed a negligible risk study by the human ethics committee of the University Hospital of Padua and granted approval via a fast-tracked review process. All participants provided written informed consent. The study was reported according to the extended Consolidated Standards of Reporting Trials (CONSORT Extended) reporting guidelines for health care simulation-based research. The trial protocol and statistical analysis plan are provided in Supplement 1.

Design, Setting, and Participants
This was a multicenter, simulation-based, 3-group parallel randomized clinical trial. The study was carried out between September 2020 and December 2021 at 4 Italian University Hospitals (Padua, Florence, Rome, and Novara), with analysis performed between January and June 2022. Residents in pediatrics, emergency medicine, and anesthesiology were recruited after providing written informed consent. Adult and pediatric Basic and Advanced Life Support-certified residents, following the AHA 3 or the European Resuscitation Council recommendations, 4 were eligible. Only Pediatric Advanced Life Support (PALS)-certified participants were eligible for the team leader role. Exclusion criteria were participation in the previous pilot study of the PediAppRREST app, 11 or personal leave during the study period.

Interventions
Participants were stratified by hospital location and residency specialty. Among PALS-certified residents, 1 participant was randomly assigned to each team with the role of team leader. Among the remaining PALS-and non-PALS-certified residents, 2 participants were randomly assigned to each team to obtain teams of 3 members, with at least 1 PALS-certified resident for the team leader role.
Teams were block randomized to 1 of 3 study groups in a 1:1:1 ratio using a computer-based sequence and sealed envelopes. All the teams conducted the same scenario of pediatric cardiac arrest using a different cognitive aid according to the randomization allocation: the PediAppRREST app (PediAppRREST intervention group); a paper-based cognitive aid, the AHA PALS pocket reference card (PALS control group); or no cognitive aid (null control group). The standard simulation was a 10-minute scenario of nonshockable in-hospital pediatric cardiac arrest 3 conducted in an off-site setting (eAppendix 1 in Supplement 2). A simulation team member played the role of a nurse and acted according to a predetermined script. 15 The manikin (Resusci Junior QCPR; Laerdal), the environment recreating an emergency department resuscitation room, and the study procedures were standardized at each study site. Further information about the study procedures and methods have been described in the trial protocol in Supplement 1 and elsewhere. 16

Outcomes
Two independent and trained reviewers (G.T. and E.F.) 17,18 with clinical and simulation expertise assessed study outcomes by reviewing the video recordings of all the study scenarios. Disagreements were resolved by a third reviewer (G.M.). Interrater reliability between the 2 reviewers was measured.
Participants and outcome assessors' blinding was not possible due to the nature of the interventions.

Primary Outcome
The main outcome of the study was the number of deviations from PALS guidelines during the management of pediatric cardiac arrest, as measured by a 15-item checklist. For each item, a score of 1 point was assigned if a deviation (ie, delay, error, or omission) occurred, while zero was assigned if the action was correctly and timely performed. Scores ranged from 0 to 15, with lower the scores indicating fewer deviations from guidelines. 16

Statistical Analysis
Sample size was calculated considering the results obtained during our previous observational 12 and pilot 11 simulation-based studies, in which we found a within-group SD of 2.2. Each study group needed 29 teams to identify a difference of at least 3.0 points in the error score (ie, 20% difference in the primary outcome measure) using the Tukey-Kramer (pairwise) multiple comparison procedure at a 5% significance level and 80% power (PASS sample size software version 11; NCSS). In consideration of possible technical issues, we planned to boost the sample size by 20% per group, aiming to include 35 teams in each group, with a total of 105 teams (315 residents).
The results were summarized with counts and percentages for categorical variables and mean and SD for normally distributed quantitative variables. For variables with asymmetric distribution, we also reported the median and IQR. The data were analyzed with generalized linear models considering the analysis of variance for quantitative variables, binomial distribution for binary data, generalized logit for multinomial nonordinal data, and cumulative logit for multinomial ordinal data (eAppendix 2 in Supplement 2). In case of statistical significance, pairwise comparisons were performed adjusting the P values and the 95% CIs for the primary outcome with the Tukey-Kramer method to take into account the multiplicity of the comparisons. P values were 2-sided, and P < .05 was considered indicative of statistical significance. Agreement between reviewers for error scores was calculated on primary outcome measures through the CCC and its 95% CI calculated with the bootstrap method considering 2000 resamplings. 31 All the analyses were performed considering the intention-to-treat principle (participants analyzed as randomized); a per-protocol analysis (excluding participants with protocol deviation) was also performed for the primary outcome. The statistical analysis was performed with SAS statistical software version 9.4 (SAS Institute) for Windows.
Analyzing the individual items of the score, significant differences in the intervention group compared with the control groups were detected for CPR board positioning, call for help, first epinephrine administration, and second epinephrine administration ( Table 2). The interrater reliability for error scoring was substantial 32 (CCC, 0.960; 95% CI, 0.925 to 0.976).
Analyses stratified by residency programs showed that in each residency group, the mean           (Table 3;

Discussion
In this randomized clinical trial, we analyzed the effect of the use of the PediAppRREST tablet app on the management of a simulated case of pediatric cardiac arrest and found that its use was associated with fewer deviations from guidelines and a better team clinical performance compared with the use of the AHA reference pocket card and no cognitive aid. Given that deviations from recommendations are still frequent and are proven to affect clinical outcomes, 5 our findings highlight the potential of our interactive multimodal tablet app to improve the management of pediatric cardiac arrest and hence contribute to increased survival of children experiencing cardiac arrest.
The detected benefits of using the PediAppRREST app in our study could be related to its ability to provide multimodal interactive stepwise decision support to the team leader. In contrast, paperbased aids are static and usually display the entire algorithms in a 1-page format, requiring users to independently identify the correct steps in the algorithm and rapidly move through its decision nodes and treatment recommendations. This process can cause a mental overload in the users and in turn delay actions and increase management errors. Given the low complexity of the PediAppRREST cognitive aid, needing only a tablet as hardware support, it could potentially be easily implemented also in lower-resource settings. In addition, practice with the app through mental simulation could be facilitated and promoted by its download on personal smartphones.
A 2022 meta-analysis 10 that included mostly adult cardiac arrest scenarios and 2 pediatric studies 34,35 reported that the use of any cognitive aid (electronic or paper-based) was associated with a better team performance compared with teams that did not use any cognitive aid. However, the use of an electronic cognitive aid was associated with a better team performance compared with     a paper-based cognitive support. 10 These findings partly align with our study results, which showed a better resuscitation performance in teams using our electronic cognitive aid, but no difference between teams that used the paper-based aid and those using no cognitive aid. This divergence could be explained by differences in the cognitive supports tested, study designs, outcome measures, and scenarios.

JAMA Network Open | Pediatrics
To our knowledge, only 1 other study has compared the effect of an electronic (smartphone app) cognitive aid with a paper-based aid and no cognitive aid, and it was conducted using adult case scenarios. 36 Although different from the cognitive tool we developed, Donzé et al 36 similarly described a significantly better performance in the teams that used the electronic cognitive aid compared with the control groups. In addition, the comparison between the 2 control groups showed no significant differences in the management of the cardiac arrest scenario, 36 as reported in our study.
With respect to the quality of CPR, we found that the use of the app did not impact chest compression quality metrics. However, this cognitive tool was not developed to support the compressors during CPR, thus an improvement of the quality of compressions was not expected.
Different real-time feedback devices and a CPR coach have been proven to effectively improve the quality of compressions. 3,7,26,37 These strategies could be easily integrated with the use of PediAppRREST to further optimize the management of cardiac arrest.
Overall, our study found that the use of the PediAppRREST app globally improved the management of pediatric cardiac arrest, mostly by reducing omissions and errors that can impact clinical outcomes. The use of our app was associated with a higher proportion of ROSC than control groups. However, in our study, ROSC was achieved when predefined tasks were correctly completed during the management of the scenario. For this reason, while ROSC is an important outcome in real-life studies, it was not included as an outcome in our simulation-based trial. 10 Most of the times to perform critical actions for resuscitation did not significantly differ between the study groups. This could be related to the limited prior exposure to the app by the team While any cognitive support could potentially increase users' workload, as it may distract from the management of the scenario by requiring extra attention and concentration to understand its use or follow its guidance, we found that the use of PediAppRREST was not associated with a higher perceived workload among users, and its usability was rated as good. 33 The novel cognitive aid was also reported to have a lower mental demand compared with both control groups, similar to other studies of cognitive aids. 10,33,34,37,38 This could be related to its ability to ease the cognitive burden of some tasks (eg, calculating drug doses, follow the correct sequence and timing of resuscitation actions, remembering the possible reversible causes and their treatments). Our study also offered an opportunity to collect participants' issues and suggestions, which were then used to optimize the app (ie, by adding the flowcharts in a section of the app) and its use (ie, by choosing a dedicated team member to use the app, different from the team leader), through a continuous iterative process encouraging personal users' preferences, with the purpose of individualize the use of the app and improve its usability.

Limitations
This study has some limitations. The main limitation is that only residents were included as participants. We elected to include residents because they are frequently the frontline physicians managing the first minutes of a pediatric cardiac arrest in many institutions. 10,34,35,[39][40][41][42][43][44] In our study, resuscitation teams were comprised of only 4 members (3 residents and 1 actor playing the nurse role); had we recruited a larger team, including attending physicians and physicians from different settings, this may have potentially influenced our outcomes and the usability ratings of the app. The second limitation is that the error score used to assess the primary outcome has not been extensively validated. However, none of the available validated tools was comprehensive enough to include all the deviations from guideline recommendations that were identified during previous observational studies [12][13][14] and that could impact on clinical outcomes. 5,45 A comprehensive tool was needed to accurately assess the possible impacts of the use of the PediAppRREST on the management of pediatric cardiac arrest. In addition, in October 2020, the AHA guidelines were updated 3 ; thus, we verified that the content of PediAppRREST, and outcome measures were consistent with the new guideline recommendations. 3 The paper-based cognitive aid used by the PALS control group was changed from the AHA 2015 PALS pocket reference card to the new version of the tool released in October 2020. Due to the nature of the intervention, participants and outcome assessors' blinding was not possible, and this could have affected results, although we ensured blinding of the study statistician. Additionally, this is a simulation-based rather than a real-life study, which would be extremely challenging and ethically not justifiable to conduct in the emergency department setting due to the low frequency, unpredictability, and complexity of pediatric cardiac arrest events. Thus, caution is warranted before our results could be extended to clinical practice.

Conclusions
In this randomized clinical trial, teams using the PediAppRREST app had fewer deviations from international guidelines and a better clinical team performance during the management of simulated